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"Here's Looking at You" (HLAY) is a creative 
prevention curriculum designed to cultivate a commitment among young 
people to deal in responsible ways with alcohol in their environment. 
The model, developed as a demonstration project for the National 
Institute on Alcohol Abuse and Alcoholism^ consists of self-contained 
teaching units for grades K-12. This non-technical report, directed 
to non-research professionals, describes an extensive 3-year 
evaluation, conducted to measure the effect iveness of the program in 
grades 4-12. Three Seattle school districts and two Portland school 
districts participated. Some schools were assigned to the 
experimental condition, and comparable schools were assigned to a 
control condition. The test instruments were administered to some 
students both before and after exposure to the HLAY curriculum; 
students were followed for 3 years. Immediate results showed a 
definite curriculum impact for all students in the area of 
about alcohol and alcoholism. In addition , self-esteem was favorably 
affected for those in grades 5, 6, and 7. Decision making skills were 
improved for students in grades 6-12. Attitudes were least affected, 
although there was evidence that students in grade 8 and younger 
students changed somewhat toward favoring moder at?: drinking as 
opposed to excessive drinking. The program appeared most effective 
intermediate grades, arguing for early intervention . Other 
influences, particularly parents^ peers, and religion appeared to 
have a greater impact. The report contains a number of graphs 
illustrating the evaluation process and the results of the study. 
(More complex data is contained in the Scientific Appendix and the 
Hanual of Evaluation Guidelines which are available from the Social 
Research Center, Washington State University. ) (JAC) 
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R EAD THIS FIRST i , READ THIS FIRST! 



PREFACE 



It is difficult to prepare a project report that will meet the heeds of the 
varieties of potential readers. Program and Curriculum planners, teachers; and 
others contemplating the adoption of this kind of prevention program for their 
young people need to know* as simply and quickly as possible, "the bottom line" 
of what the evaluators have learned: Does the program in question work? To what 
extent, and in what ways, has the program proved capable of achieving its intended 
objectives? The lengthy, complex, and technical process by which the answers to 
such questions were obtained by the researchers is understandably a matter of 
secondary interest to non-researchers. The present report is prepared with this 
reall ization. 

Those among our readers who want to know, in a period of ten minutes or less, 
our general findings about the impact of the "Here's Looking at You" Alcohol 
Education Program should skip immediately to the CONCLUSION (pp. 36 - 39). Those 
who want more information about the basis for our conclusions, still in fairly 
simple form, however, will wish to read some or all of the rest of the report. 
It is intended for non-research professionals in school systems, in community 
prevention projects, or in other organizations and agencies chared with finding 
modes of intervention to prevent or reduce the incidence and severity of alcohol- 
abuse among school -age children. Since the report deals only superficially with 
the research methodology employed, or the statistical analyses lying behind our 
findings, the reader is, in effect, being asked to assume that the evaluators 
did their work competently and objectively. A relatively short and simple report 
like this one, however, may be somewhat deceptive, for it cannot possibly convey 
the complexities, the time, or the anguish of the three years' labor from which 
it derives. 

Readers who would 1 ike to know more about such matters are invited to send 
for certain other documents from our project files. First of all, there is the 
Scientific Appendix to this report, which may be obtained by sending a written 
request and twenty dollars ($20.00) to the Social Research Center, Washington 
State University, Pullman, WA. 99164. This Appendix covers the questionnaire 
items comprising our instruments, and the frequency distributions for each item; 
how we constructed our scales and other variables; descriptive statistics (means , 
standard deviations, etc.) for our major variables, both in our cross-sectional 
and in our longitudinal samples; the basic comparisons lying behind our assertions 
about curriculum impact; the theoretical model and analytic strategies guiding 
our ongoing analyses; and certain other 'matters of scientific interest. Another 
document of potential interest to readers of this brief report is our Manual of 
Evaluation Guidelines for the "Here's Looking at You" Alcohol Education Program, 
nicknamed our "cookbook . " This document can also be obtained from the same 
address for another $20,00. It is intended for on-site evaluators with at least 
some experience, but not necessarily with any statistical sophistication. The 
"cookbook" is a step-by-step guide covering all the evaluation procedures for 
the program on a 1-year (or year-to-year) basis, without any longitudinal com- 
ponent. It begins by discussing arrangements with schools and parents for 
implementing and evaluating the program, and then goes into data collection 
planning, procedures, and instruments, the codi ng of the instruments, and the 
basic procedures and formats for analyzing and reporting the data. Copies of 
the test instruments for each grade level are included, along with other samples 
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ILl^lVti ? f e val " a JTon materials. Still other documents from our files 
end of this report ^ *™ U * t&d 1 " the ^ography at the • 

,>c Q J h °-! Wh ?-v 1sh f° know more about the "Here's Looking at You" program 
nil write Jul ^^P^'/ 3 ^^, teaching methods , or teacher training! should 
not write to us but instead to Roberts and Associates, 9131 California Avenue 
S&lSfl* 3 * WA 9 ? 136 ', {Phone: 206-932-8409.) Our work has to do only with 
ml rPnn,t re ^ ar J h 4 h and ^ 0 f^who wantjnformation, beyond what is confined in 
£S0 r ! PP iS* a ^ ut the methodological, technical, or statistical details of our 
work should write to us for the Scientific Appendix, the Manual of Evaluation 
Guidelines, and/or other documen ts listed inTur Bib liogr aphy? bValuat1 Q" 

A. L. M. 
R . H . H . 
R. A. W. 
K. A . K. 

Pullman, Washington 
November, 1981. 
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1. 



INTRODUCTION 



Background 

Our relationship as evaluators. to the "Here's Lookinq at You" Alcohol 
Education Program (hereinafter HLAY Program) goes back to the late Pall of 
1975. Earlier that year, the Health Education department of the Educational 
Service District No. 121, Seattle, Washington, had received a 3 -year grant 
from_ the National Institute on Alcohol Abuse and Alcoholism (NIAAA) to create 
and field- test a model alcohol education program as one of several NIAAA 
demonstration projects. The principal investigator and creator of the 
HLAY program at ESD-121 was H. Clay Roberts, and chief among his several 
collaborators on the project was Carol Mooney. They produced what is, in 
our opinion, a highly creative and imaginative prevention curriculum derived 
rrom the most advanced theoretical assumptions current among health educators 
during the 1970s. The ultimate objective of the curriculum is to cultivate 
a commitment among young people to responsible ways of dealing with alcohol 
in their environment. This includes tneir decisions in alcohol -related 
situations, such as riding with someone who has been drinking, as well as 
their personal drinking behavior. In this context, "responsible" decisions 
would include both abstinence and light-to-moderate drinking, depending 
upon the values of the person, but would exclude heavy drinking, drunkenness, 
and other choices which might lead to anti-social or self-destructive con- 
sequences. 

In accordance with the terms of their NIAAA grant, Mr. Roberts and his 

colleagues at the ESD called for bids on an evaluation contract, and at length 
concluded one with Us at Washington State University (WSU) , after the HLAY 
program had been created and was ready for implementation. That contract and 
its renewals lasted until the Spring of 1978. It required us first to evaluate 
the teacher- training component of the HLAY program, in two phases, between 
which certain modifications and "fine-tuning" were carried but by the ESD 
educators to enhance the effectiveness and the efficiency of the teacher 
training. Also evaluated was the usefulness of the "multiplier effect" in 
teacher-training— that is, the reliance on ESD-trained teachers to train fellow 
teachers back in their respective school buildings. Finally, under our 1975- 
1978 contract with ESD-121, we created and pilot-tested a research design, and 
some research instruments, for evaluating the HLAY program in the classroom for 
grades 4 through 12. This was a minimal research effort intended to get pre- 
liminary information about actual teacher performance* as well as about the 
Impact of the new program on the students themselves. Additional information 
about these early evaluation efforts and outcomes may be obtained from the 
publications and project reports listed here in our bibliography under Rankin, 
et al., 1978; Tarnai, et al., 1978; Mauss, et al., 1980; and Tarnai, et al . , 1981. 

The next phase of our work, from 1978-1981, is what is covered by the present 
report. In mid-1978, we at WSU received a grant from NIAAA to conduct an exten- 
sive and thorough 3-year evaluation of the HLAY alcohol education program. This 
new grant called for a kind of reversal in our relationship with the program 
developers at the ESD-121: We, and not they were to carry the initiative 
in the decisions about the dissemination and evaluation of the program. These 
decisions would be based upon our quasi-experimental research design, which 
designated those schools and classrooms at the various grade-levels that would 
serve as experimental or as control groups; Our ESD-121 partners would train 
the teachers and disseminate the teaching materials in the schools that we 
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targeted. Their partnership arid consultation in the research endeavor were 
aft»^ ? lZ essential, so they too were- given a new NIAAA grant in mid-1978 to 
T££$rh U * n ^£t Qnt ll e en terprise, however, was now defined as primarily 
a research project, rather than as a program demonstration project; 

The Nature of the Product Being Evaluated 

U ■im^'°Sf?2^! i -"*-^E? , 5 Looking at You " Alcoh °l Education Program 
hS at cultivating "responsible" ways of dealing with alcohol on the parts 

of yocng people. As far as personal behavior. is concerned, for some of the 
youth, responsiole in this context may mean total abstinence, depending* upon 
religious or otter personal values. For those who choose to drink alcoholic 
beverages, however, the operational meaning of "responsible" will be a mode of 
or nt^r"7nH]rL?onc 0 n/ r ; rtk - nd behavior that is free of problematic consequences 
^nul- fr iZ n - lc ? tl0n ? of abuse ' Sponsible decisions and demeanor will also 
InS UL c 6 " 15 ? V f- ?S appropriate efforts to discourage abuse among friends 
iiiiiiS? ii.^ P --i"i - in the tne oretical basis of the HLAY curriculum is the 
assumption that abusive or problematic drinking behavior is attributable to 
many factors, including alcoholism; ignorance about the drug alcohol and its 
physiological or psychological functions; low self-esteem; difficulties in 
using decision-making skills; and difficulties in identifying and applying 
I!S2 P II a ?i;~ P f M strategies. Accordingly, the curriculum contains components 
aimed at enhancing knowledge, self-esteem, and skills In coping and in decision- 
making. The ultimate goal of the entire alcohol-education package, of course, 
is to impact actual behavior where alcohol is concerned. 



among the existing curricula in health education, in biological science, in 
SUll f^ ies * H ln 5 0m e economics, or in nearly any other traditional subject- 
52 tje necessary teaching materials for the curriculum have 

been prepared and boxed up in specific grade-level kits, which include games, 

iJrt^jSi*"!"--*?"" kinds ' as we n as the substantive teaching guides 

and textual materials. Three full days of teacher training are normally required 
t°Z adequate preparation in the classroom use of these materials. Readers who 

program it ^L" TnT^tTJ I' he "Here's Looking at You" alcohol education 

program, its kits and materials, the teacher training, and so on, should make 
direct contact with Roberts and Associates (see Preface). 

THE EVALUATION DESIGN AND DATA 

™ii T 5 e / es earc h design for our evaluation, and the nature of the data we 
collected, if treated thoroughly here, would require a lengthy and complex 
discussion that we assume would hold little interest for most of our readers. 
S JSi.K C 2Si3fsS 9 0 J lea 9«es in the research community who would like the benefit 
lid „ 3 ; nfcrn, ation are invited to send for the Scientific Appendix to 
this report, which can be purchased separately, as we hav e explained in the 
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The- Basic Design 

Pp r the benefit of the general professional reader, we will explain briefly 

that we used what program evaluators usually call a "quasi-experimental" design, 
we assigned some schools to an "experimental" condition and some comparable 
schools to a "control" condition; or, as an occasional variation, some classrooms 
within a school to the experimental condition, and other classrooms within the 
same school to a control condition. Our tests or questionnaires, designed to 
measure the effectiveness of the HLAY program, were administered to the "experi- 
mental" students after they had been exposed to the curriculum for a given school 
year, while the same instruments were administered to the "control " students 
without their having been exposed, or before they were exposed. Some of our 
experimental students were deliberately administered the test instruments both 
before and after exposure to the HLAY curriculum (pre-test and post-test) to — 
allow us to assess the actual a priori comparability of our experimental and 
control groups. 

Altogether, five different school districts participated in our evaluation 

effort: three in the Seattle area and two in the Portland area. Some of these 
districts were large and varied enough to provide us with both experimental and 
control students; others provided experimental students only, or else experimental 
students at some grade levels and control students at others. We del iberately 
chose school districts that would provide us with as much variety as possible 
across the entire project, including small and large districts, rural and sub- 
urban districts, urban schools with and without much ethnic variety, and so on. 
Naturally, we had to take the students, classrooms, and schools as we found them 
already constituted by the district administrations, so we were not able to 
assign the students randomly to the experimental or the control conditions. 
Nevertheless, we did succeed in negotiating with each school district as much 
leeway as feasible in making random assignments of schools and/or classrooms, 
and we sought for as much similarity or comparability as possible between our 
control and experimental schools and rooms. 

A particularly valuable feature of our evaluation was its longitudinal 
component, in which we followed many of our control and experimental students 
across all three years. Depending upon when certain students entered or left 
our project, and upon curriculum variations at certain grade-levels or schools, 
some of our experimental students were exposed to the HLAY Drogram only one of 
the three years, some were exposed two years, and some all three years. A one- 
time exposure could have occurred, of course, in any of the three years . A 
two-time exposure could have occurred either in years one and two, in years two 
and threo, or in years one and three. Whether exposed or not, students whom we 
tested in any given year were followed up whenever possible and tested during 
the subsequent years of the project as well. Our research design, interacting . 
as it did with the many natural contingencies and changes in the school populations 
across time, was enormously complex, a situation compounded by the use of a 
system of self-generated private identification codes, designed to protect the 
anonymity and confidentiality of student responses on our test instruments 
(Kearney, 1982). Nevertheless, all of these complexities had to be accepted, 
for they enabled us to assess the impact of the HLAY curriculum not only in 
the jmroediate sense, after only one year's exposure, but also in the longer 
run and after varying degrees and modes of cumulative exposure. Furthermore, 
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?^« L nJ ™P ac J on actual drinking behavior could be assessed only after some 
lapse of time beyond the exposure itself" — «»weu omy arter some 

.The Nature, Quality, and Quantity of the Data 

S | recularl ^^"^tal students received this tes? training f 
uunng me regular 3-day training workshop called for in the hi ay R mnrJ 



(No testing was attempted below the 4th grade berati** 3 ISLSrS . S ! Parately 



--.-» _. 3 icvci.; me test-retest reliability eoeff icipni-s raimiatoH 
arter pilot- test ng ranged mostly from .50 to .90 f or the var ou s Uems in our 
instruments. Reliability coefficients falling below that range tended to occur 
^Ri« he f unqe : c^ldren and with attitude measures, which seemed the least 
stable. Among the most stable reliable) were the self -reports of drinking 
and drug-using behavior. Where we constructed compos te measu res based u Son • 
more than one test item, the al£ha reliability coefficients we^ thl faroer 
of course, the more the test Ttfe that were included. The construct vl??dity 

WwJSbn'fiTllirfiSja t0 , th t Hl - AY curriculum was usJlll^SSmSS tfirlugh 
h«* 1th educators at ESD-121. The validity of the test item? 

not related to the curriculum Was usually established either internally or hi 
reference to their successful uses elsewhere in the professional H?er^tu re 7 
(More on validity and reliability can be found in either ?he Ma nui or the 
Appendix mentioned in the Preface to this report.) or tne 

The contents of the tests at the various grade levels depended in Dart uoon 
level° nt OtherS se^thevlinded 11 « hic ^^ ^pectively per?Iined at ZEtjgS* 
to the riixt that is ^ t!^\, haVS a . cumulative quality from one grade-level 
11*121 2 6 r the * est at an y given grade level tended to include or 

repeat the measures that had appeared at the earlier grade levels In the iunior 
and senior high grades, the HLAY curriculum and accompanying tests were given at 
whichever specific grade-level the school administration selected- that I 
|1^9 r ^e 7 °r 8 or 9 for junior high, and either grade 0 or 11 o> 12 for 
senior high (though sometimes grade 9 was included at the senior high level) 
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The^factors measured by our instruments at the various grade levels, including 
botb curriculum-related and other kinds of measures* are indicated by the summary 
in Table 1. Single copies of our test instruments at each grade level may be 
obtained free of charge by writing to the Alcohol Education Research Project, 
Social Research Center, Washington State University, Pullman, WA 99164. 



Table 1 

Variables Measured by Student instruments* 

Variable . —r 

Personal Code 

Demographic Variables 
Gender 
Family size 

Birth order 2 
Age x 

Ethnic group * 

_ *> 

Knowledge* 

x 

Self-Esteem* 

A 

Decision-Making* 

Assigning problem responsibility 

Generating alternatives. x 
Selecting responsible alternative x 
Assessing advantages and disadvantages x 

Attitudes about Alcohol* 

Total attitude score x 
Alcohol for mood enhancement 

Tolerance of abstinence x 
Intolerance of abuse x 
Tolerance of moderate use x 
Influence of others 

Alcoholism as disease, not character defect 
Treatment of alcoholics 

Childhood Drinkinq Behavior x 

Adolescent [Drinking Behavior 
Irresponsible uses 
Problem drinking 
All drugs 

All drugs but alcohol 
Current drinking situatipn 
Expectations about drinking 
Monthly frequency 
Yearly frequency 
Quantity 

^monthly 

QF yearly 
BAC level 



Grade 



_ 


R 

\J 


on - 


jn 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 






X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 



X 
X 
X 



X 
X 
X 
X 
X 
X 
X 



X 
X 
X 
X 
X 
X 
X 
X 



X 
X 
X 
X 
X 
X 
X 
X 
X 



X 
X 
X 
X 
X 
X 
X 
X 



X 
X 
X 
X 
X 
X 
X 
X 
X 



X 
X 



X 
X 
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Table 1 (continued) 
Variables Measured by Student Instruments* 

Variable • Grade Level 

- Variab '^ 4 a 6 JH SH 



Attitude Toward Alcohol Education 

Peer/Parental Influence 
Peer support 

Peer control « x x x x 



Peer drinking influence x x x x x 

Parental support x x 

Parental control x x x x x 

xx 
x 

x x 



Peer loyalty x x x 

Parent versus peer x 



School Variables 

Extracurricular activities 



Level of performance x x 

<;ati<f^ti«n X X X X X 

X X X X X 



Other Non-Curriculum Variables 
Religious control/constraint 

Conventional commitment x 
Smoking behavior x 



x 
x 



General health " x x 

General happiness x 



x 

x x 



*N0TE 

a I r C nrdJnS°^ a "h t o t0 n emph ! S1 'f- ^ table organizes the curriculum variables 

according to the conceptualizations of the researchers . not necessarily of the 

SKJSfS d6Ve ° P r themsel y es - ™e dif ferences betw een thltwo caribe 
SmsJs? f mai y * t0 sorne , evolutlon in the thinking of both parties after their 
aSSES -ag^emejt Upon the construction of the test instruments, and also to 
different conventions in the professional literatures, respectively of social 
the e two e r nS ?^ ^-th_education. In any case, the differences between 

tlon iLul "SfSSS 1 f^ 10 ? ? ! r f T 9reat ; Both identif y Pledge or informa- 
2 ShJE hit ■ -- d alcono abuse, self-concept, and decision-making skills 
as three of the major curriculum components. The main difference lies in the 
fourth component, which the curriculum people identify as coping (defined as 
nn S ^f h3t ° ne knows to feel better" ) . The researchers haTeTns lead focused 
O" |^odes. ii as the fourth major component (usually meaning dispositions toward 

one's env?rnnJn,? PP °K d * V™?™^}*" ways of dealing with alcohol in 
one ^environment). The curriculum people, on the other hand, see attitudes 
M a separate^component but rather as permeating the entire processes of 
rlnlrVn » °- ^ci s i on-maki ng . Thus, some of the test items that might be 
a r c ^h" - Ca 2- S Jf ""PlOfl" or of "decision-making" by the health educators 
IhLya *S Tfl f d J 1n , this table under attitudes , and others perhaps elsewhere. It 

t c n ° ed that our self-concept variable is made up essentially of 
HB? 3S5rii S^P^P (f ° iow1n g CPopersmith). This overlaps with, but is 
not identical to, the three dimensions of • sel f-concept now identified by the 
curriculum developers as "self-awareness," "self-assessment," and "self- 
change." (See Figure 1) ~ 
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Figure 1 

COMPARISON AND RECONCILIATION OF THE RESEARCHERS 1 AND THE HEALTH EDUCATORS' 
CONCEPTUALIZATIONS OF THE "HERE'S LOOKING AT YOU" CURRICULUM 1 



II. 



Researchers 1 Model 

KNOWLEDGE AND INFORMATION 2 

A* Traditional reasons for drinking 
or not drinking 

B. Physiological and pharmacological 

properties of alcohol 

C. Estimating dosages and their 

consequences 

D. Problem-drinking* alcoholism, 

and community responses 



SELF-CONCEPT (SELF-ESTEEM) 3 

A. Self-awareness — — " 

B. Self-assessment 



ii. 



III. DECISION-MAKING SKILLS 



A. 
B. 

C. 



D. 



Assigning problem responsibility 
Generating alternatives (e) 
Predicting advantages arid dis 
advantages of selected 

alternatives (c) 

Selecting a responsible al ternative 
(c) 




Health Educators' Model 

INFORMATION (KNOWLEDGE) SKILLS 

A. Gather a Body of Knowledge 

(including an understanding of 
various areas, viewpoints, 
sources, 1 imitations, and 
resource people) 

B. Evaluate Information (including 

a consideration of the source, 
the reliability, accuracy, 
recency, verifiabil ity , rele- 
vance, adequacy, conflicts, 
or ambiguities) , 

SELF-CONCEPT 

A. Self-awareness 

B. Self-assessment 

C. Self-change 



DECISION-MAKING SKILLS 



Identify and define problem 

Consider values, attitudes, 
feel ings , and pressures 

Consider available information 
and alternatives 

Predict consequences of alter- 
natives 

Select an alternative and act 



IV. 



IV. ATTITUDES 



COPING 

A. Understand the nature, sources , 

and effects of stress on self 

B. Identify personal coping behaviors 

and their consequences 

C. Explore alternative coping 

strategies 



A. Total (composite) attitude score 

B. Attitudes toward alcohol for mood enhancement (c) 

C. Tol erance toward abstinence (d) 

D. Tolerance toward moderate alcohol use (d) 

E. Intolerance toward alcohol abuse (d) 

F. Attitude toward influence of others (c, d) 

G. Recognizing alcohol as disease, not character defect 

H. Appreciation of need for treatment of alcoholics 



(See next page for notes to this Figure) 
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NOTES to Figure 1: 



'ac^l l^?™?^?" 8 ' M ° del " bel0w 15 a classification of what the researchers 



2 Not all four of these aspects of knowledge were measured at all arariP ip Vols 

Sjih?nl°l! d by coojerp-h. As a ;o s t-hoc conceptual sffb?t! { ?! 

(C) = of d w SJ e th« h hLf5i S 3 eaS i! re devel °P ed fa y the researchers relates to some 
or what the health educators consider "eoping." 

(d) = 0 ? d w h^ e ? h ^hl?i; S l! 635 ^ 6 devel ? ped h * the ^searchers relates to some 
of what the health educators consider "decision-making." 

t«Mi^h! 1 fl3 56 ! pparent that in the health educators' model there is not a 
total and discrete separation between coping and decision-making, especially 
native ^e^rating, selecting, and predicting the consequences of a ter- 
native choices This overlap accounts in part for the ^conceptualization made 
by the researchers for measurement purposes. ' ZKlon 
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table 2 

Total Numbers of Questionnaires Received (Pretest, Posttest, and Control) 
Pr0j " Y6ar SM^M^MGUMGrn Gr 12 Total 



1980-81 860 IIS II jg xSlI & g ftg 

TOTALS 2333 2380 2634 2425 2976 1984 980 622 1077 17 [m 

"naive"^un1x f ooseS^^d°rn^% data f ° r ^ptlyi Purposes is that based on all 

our instruments Thus tht «»£ I M 1 ^" 1 - education curriculum or tested with 
our instruments. mus s the data from these naive students permit deserintinn nf 

the* distribution! of U th\ nt Ln S We f ° U P ^> ? r0Vidin 9 nonnal^e inform ^ion^about 
Sler IS Slth thf r^ n >M?n 6S ° n . th ! Y ariables we ^asure, in the absence of any 
J curriculum or test instruments, The numbers of cases of these 

tnrovidffr^np^^ eSent K d 1n / able 3 (9rades 10 ^d 11 have been comb?ned 
to provide a respectable number of cases at that level). 

Table 3 

Numbers of "Naive" Pretest and Control Students from Years 01 and 02 

Gr_4 Grv_3 Gr_6 Gr_7 Gr_8 Gr 9 ' Gr 10-11 Gr 12 Total 
1109 404 476 550 1125 493 262 430 -4,849 

. a . Fo [ th J assessment of immediate, short-term impact of the curriculum another 
data subset was constructed consisting of the questionnaire responses from 
ffittrgl and control students who were new'to the oroie.t'in' Years 0? nr pj 
iJtl j ^ e control data are from control students the first time they were 
tested, and the experimental data are from students who were exposed to the 

SrS^JS f ? r the fl>St time and then posttested. The resulting numbers of 
these students are presented in Table 4. 

Table 4 

Control and Posttest-Experimental Data for Assessment of Immediate, 
Short-Term Curriculum Impact (Years 01 and 02 Only) 

Gr_4 Gr_5 Gr_6 Gr_7 Gr_8 Gr_9 Gr 10-11 Gr 12 Total 

Control 699 272 331 426 765 114 116 265 2 988 

Posttest 834 366 365 215 1047 493 220 280 3,820 
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or Ibngellel^Sl? 1 '^ ctr^lum* ^3L** T 1 "!"* the Native 
tionnaires could be linked !crn« Jv «- 6 - numbe s af students whose ques- " 
mental and control ^ students! ^ bJ^Lh^f^T^ 0 ^^ 6 P^ect (both experi- 
the data subset for ^iLKtS^t^&Vftr 1 ® HHf S |t?" ? ^^ 

Table 5 

Total Numbers of Cases of Three-Year Linked Longitudinal Da*a 
— ^^LhhSI^SS^ I^n 10^12 Total 
300 266 2 26 13 62 30 0 897 

Hbwe J! wTafso M^^^X^ - relatively small, of course, 
was possible for only S T n Z more cases where the linkage 

• (both experimental and^cfntrol ) 'is Sivln fn tHi-l of h ?"ch two-year linkages 
01-to-02 linkages plus (fSr ?hose dis^HrJ, In? • 6 > wh ^h includes the year 
the year 02-to-03 TTnFaqes This dlU lit S ^? 1 lrn ? g our Project in year 92) 
examine the cumulat vflnd* longer- term wi J/Jj° even ^]^ be Jsed to' 
noted that the cases represented in Sp SKfJ- th f ^Cr 1 ™ 1 ™- It should be 
5) are a subset of the ^S^aJfe^rSr^^^gft^J.** S6t (Table 

Table 6 

Total Numbers of Cases of Two-Year Linked Longitudinal Data 
^M^horts: ^ M ^^ Mmi0iikiij ^ 

457 426 323 416 813 375 H 116 2>937 

subset fwither 0 longitudinal o^ross°sect?^?? h the ^al data set and each 
systematic biases in SS SSbsSolS wS ^^ h f eU ° C ? Vered n0 significant 
subsamples are fairly representative nfthl \ J- refore .9 ulte confident that our 
data permitted us to comoare th! JJJ» 1,2* ? nti r e Project. Nor, wherever the 
other, did we find «J consistent b?Ss£5 S^i??! " 9 - ^' stricts witfi each 
understandable difftShlS t • 6 s or differences by district, except for 

The ethnic d\ ! str ^ Hike race/ethnicity), 

combined, was appr^ma^ districts 
2% Hispanic, 7% Asian anH <!* ^ ! ack ' 3 " I^an/Native American, 

divided' between Sles 'and females' ambl 9 uous - samples were almost evenly 
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THE SHORT-TERM IMPACT OF THE ALCOHOL EDUCATION CURRICULUM 

Now that the reader has, we hope, a fairly clear idea of the design and the 
data-base for our evaluation, we are ready to review some of our results. We 
begin with the immediate or short-term impact of the HLAY curriculum, that impact 
which we could measure after only one exposure of the students to the alcohol 
education unit. We remind the reader that no immediate curriculum impact on actual 
drinking behavior eoulri fpas-ihly Be measured, given that the test instruments 
were normally administered at the completion of the unit, and that the measures 
dealt with drinking behavior in the recent past, generally before exposure to the 
HLAY . curriculum. Measures of immediate impact* then, are limited to the various - 
cognitive and affective components of the curriculum: knowledge and information 
about alcohol, etc.; self-esteem; coping and decision-making; and attitudes. 

Table 7 summarizes the results of literally months of careful comparisons 
and analyses. The various components and sub-components of the HLAY curriculum 
at the respective grade levels are listed down the left side of the table, and 
plus signs (+) indicate at which grade levels a curriculum impact was established 
unambiguously and with statistical significance (p = .05 or less). Minus signs 
indicate the few places where curriculum exposure was followed by the opposite 
of the expected results. The large number of remaining (blank) spaces indicate 
where the curriculum impact was ambiguous or not statistically significant, even 
though many times it was in the expected direction. Standards for the determination 
of unambiguous impact (+) were quite rigorous, so these results are probably 
conservative estimates of impact. Judgments were based not only on sheer 
differences between experimental and control students, or on pre-test vs. post- 
test differences, but also on considerations of comparability between control 
and experimental samples, test effeets (or sensitizing) on post-tests from pre- 
testing, and other such contingencies explained in our Scientific Appendix . The 
results in Table 7 cover all five of our project districts combined; as we men- 
tioned above, our design was not equally elaborate in all districts, but we 
found few systematic individual district differences or idiosyncrasies in our data. 

The bar graphs (Graphs 1 through 5) also provide selected illustrati ons of 
some of the comparisons that lie behind Table 7. These particular illustrations 
were selected here because they are among the most positive examples of curric- 
ulum impact. In general, the table and the illustrative graphs are quite 
encouraging in what they indicate about immediate impact. There are more 
unambiguous instances of impact for the various curriculum components and the 
various grade levels than we have ever been able to establish heretofore, either 
in earlier (and more tentative) analyses of these data, or in the pre-1978 
period of our work (see Tarnai , et al ., 1978; and Mauss, et al . , 1980). We 
have nearly always found an immediate impact on the knowledqe/information com- 
ponent, but the more affective components have always proved much more resistant 
to successful outcome and/or measurement than is the case here. The HLAY has 
clearly enhanced self-esteem after only one exposure at grades 5, 6, and 7, at 
least. Various aspects of decision-making and coping skills have also been 
improved, especially from grade 6 on up. Attitudes toward alcohol use have 
always proved the hardest to change in the short run, but at least there is some 
evidence here that students exposed to the HLAY program in grades 4, 6, 7, and 
8 were changed toward a posture of moderate use after one curriculum exposure. 
Implications of all this for actual behavior toward alcohol would presumably 
be favorable, but we will need to turn to our longitudinal data to look at that 
issue. 
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Table 7 

Summary Evidence of Unambiguous Immediate Curriculum Impact 



4 5 


_a_ 


7 


8 9 10-11 12 


+ -F 


+ 


+ 


+ +-- + + 


+ _ 









Know! edge 

Sel f-Esteem 

Decision-Making 

Assigning Problem Responsibility + + + 

^Generating Alternatives - + ~+~ — — 

^Selecting Responsible Alternative — — — _ 

Advantages and Disadvantages Z~ ~ + + — 

Attitudes about Alcohol 

Total Attitude Seore (all dimensions) 

l Mood Enhancement I z — — — 

2 Tolerance of Abstinence — — 

intolerance of Abuse — 

2 Tolerance of Moderate Use ~~T~ + +~ — — 

influence of Others _ — 

Alcoholism as Disease, not — - — — 

Character Defect 

Treatment of Alcoholics — : — - 



^able^l)^ al? ° an aSPSCt ° f copi ' nq by curr iculum developers. (See footnote, 
Considered also an aspect of decision-making by curriculum developers. 
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SBfigB-j. 

A. Immediate Curri culum Impact on 
KNOWLEDGE 

Grade 6 

10 



Range of 
Possible 
Scores 




B. Immediate Curriculum Impact on 
KNOWLEDGE 
Grades 10 5 11 

12 




Cont. Exp. Pre Post 



N's = 116* 216* 92 92 

♦These Ns correspond to their counterparts in Table 4, except for minor 
variations resulting from missing data (non-responses) on specific items 
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GRAPH 2 



A. Immediate Curriculum impact on 
SEbF-ESTEEM 
Grade 6 



&ange of 
Possible 
Scores 




Range of 
Possible 
Scores 



B. Immediate Curriculum Impact on 
SELF-ESTEEM 
Grade 8 

10 



6.56 7 '° 2 




Cont. Exp. 
N's = 698 952 
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Not statistically significant 



GRAPH 3 

A. Immediate Curriculum Impact on Decision-Making Skill 

ASSIGNING PROBLEM RESPONSIBILITY 
(Not included in curriculum before Grade 7) 

Grade 7 



Range of 
Possible 
Scores 



2.14 



1.93 









Cont. Exp. 
N's i ■ 426 215 



2.23 




B. Immediate Curriculum Impact on Decision-Making Skills 
ASSIGNING PROBLEM RESPONSIBILITY 
Grades 10 & 11 



Range of 
Possible 
Scores 




Con i. Exp. 
N's = 116 220 



2.83 



3,Q4_ 
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GRAPH* 

A. Immediate Curriculum Impact on Decision-Making Skills: 

GENERATING ALTERNATIVES 
Grade 6 



10 



Range of 
Possible 
Scores 











6.11 


7 . 33 


t 6.95, 




/y 










// 




* * 
• 

• _ • | 

* * 1 

• • j 

• 1 

• • ] 

* 1 

* * ! 


Cont. 


Exp. 


Pre f 



7.39 



N'S = 326 363 



128 123 



B. Immediate Curriculum Impact oh Decision-Making Skill: 

GENERATING ALTERNATIVES 
Grades 10 & 11 

10 



Pos s ib le 
Scores 



6.59 



6.79* 




N'S 



Cont. Exp. 
116 220 



7.04 



7.39 
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Not statistically significant 
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GRAPH 5- 

A. Immediate Curriculum Impact on Attitudes:* 
TOLERANCE FOR MOOERATE ALCOHOL USE 
Grade 4 
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Range of 
Possible 
Scores 



Ok, 



2.66 



2.76 





Cont. Exp. 
N's = 683 805 



2.63 



2.88 




S. Immediate Curriculum Impact on Attitudes:* 
TOLERANCE FOR MODERATE ALCOHOL USE 
Grade 8 



Range of 
Possible 
Scores 



0 



2.85 





. Exp. 



2.88 



3.0fr 




0 
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Pre Post 
N's = 756 1035 256 256 

♦Considered by curriculum developers as an aspect of the 

2? 



ion-making process 
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CUMULATIVE AND LONG-TERM IMPACT OF THE CURRICULUM 

iSb ac t S «f «£ P 21!^ ' -hW data sets that will allow us to assess the 

£5S n,! ? W - CUrriculum eith6r °ver a 2-year span or over a 3-year span or 
J?**: . Our analysis so far has been limited to the 3-year data, and wfare 

aooearinc Jn" gS? ?? -° f thl L reDort that cumulative curriculum impac? not 
taS^afLr a ?Fnf/nr^ W °- ld hGt appeared in two. With measurements 
I - PS of three years » we also have logical grounds for drawina 

some conclusions about eventual impact on actual drinking behavior? as we 1 as 

?s??c ^ U addr^HV 0n ^ he H f^ er .-^ cognitive and affective) Sn character- 
istics addressed by the HLAY curriculum. 

in thS^L 8 pr ? vfdes a summar y of curriculum impact sustained over three" years 
in three age cohorts: one which started in the fourth grade in the first year 
4 5 6 and e °J e t i?-^ sixth grade by our third year (here caT edlhe 

tt ltr SS2l2t. 2h/Sn? ^S reSS6 2 t u hr0Ugh grades 5, 6, and 7 during the 1 i f e 
or our project, and one which moved through grades 6, 7, and 8 We were not 

SJ?H?«S- n H CeS - u to u enougn cases of 3-year 1 inked data ?or any one cohS^ 
ending above the eighth grade to make feasible the analysis of Sr??tufSS Sba-t 
at those higher grades. This outcome is partly the result of theTeUtive v 

?s re the C hich1Ihonu e ^ ated ^l?^™ 

is, the high schools in our project generally installed the HLAY program in 
nnVL^T^l* in Qiti l er grade 9 SL grade 10 or one of the higher grades but 
^t'thr^t^derfnH^K*- ^h f'hool grades" However, since we have learned 
wen L 6 and behavior of young people toward alcohol are likely to be 

S^^Sv^laTyS^ jUni '° r hl ' 9h P6ri ' 0d ' ° Ur data d ° "^/probably 

"onriJ h M P^J! 19ns ( ! } on TaDle 8 indicate where the three cohorts respectively 
ended up" with regard to knowledge and information about alcohol self-esteem 
decision-making .kills, attitudes, and behavior. Again depending upon how mtny 

i^y X n"?nh reS thS H ri0US Cl ^ rri ' cula ealled *>r ^ring themiddl IschSol or * 
junior high years, the experimental students in these cohorts had differential 

K: n ? °L 6 n P n SU T ^ 4-5-5 cohort was exposed three times (once in each 
of the three project years); the 5-6-7 cohort was exposed at least the first 
two years and some of them all three years; and the 6-7-3 cohort was exposed 
laTffFn " y Iii" S ^ree (an XOX pattern) or in years two and three (an OXX 

pattern). All students, whether experimental o~control , in all three cohorts 

sta r tist a c a U lfv ? ha V' S> SS* 311 three y6ars - The signs indicate ' 

Ihf t JlfZll* significant differences between the experimental and control students 
th ' J " d rema med) in the cohort in or by the third year. On some of 

in aliTSSi SJif ns i '° ns -'-? uc h W knowledge, there was evidence of program impact 
ILVI thr ee years. On some of the other dimensions, it occurred during the 

or Se ea irj^- r ^- ai ii d lnt °J he th ! rd - 0n still others, the impact appeared 
(or beeame statistically significant) only in the third year. 

iiss^Ji*?^* 1 -' T able 8 is not as encouraging as was the earlier table on 
?S3?Sil? C " r n CU I Um - lmpaet - Aside from the knowledge dimension, which almost 
routinely yields to instruction, the other plusses on Table 8 are few and far 
^tweer,- Where the 5-6-7 cohort is concerned, there are no more plusses at all 
which may be partly attributable to the fact that some of the measuring devices 
used in the instruments (tests) are somewhat different, starting in grade 7 
than they had been in the earlier grades. On the whole, though, we are inclined 



23 



18. 

TABLE 8 



SUMMARY EVIDENCE OF CUMULATIVE CURRICULUM IMPACT 
BASED ON THREE-YEAR LONGITUDINAL SAMPLES 



Knowledge 



Self-r Esteem 

Decision-Making 
Assigning Problem Responsibility 
"Genera ti ng Al ternati ves 
[Selecting Responsible Alternative 
1 Advantages and Disadvantages 

Attitudes About Alcohol 

Total Attitude Score (all dimensions) 
'Mood Enhancement 
^Tolerance of Abstinence 
^Intolerance of Abuse 
^Tolerance of Moderate Use 
influence of Others 

Alcoholism as disease, not character defect 
Treatment of Alcoholics 

Drinking Behavior 
Irresponsible Uses 
Problem Drinking 
All Drugs 

All Drugs but Alcohol 
Current Drinking Situation 
Monthly Frequency 
Yearly Frequency 
Quantity 
QF Monthly 
QF Yearly 
BAC Level 



4 ' 5 - 6 Cohort 5-5-1 Cohort 6-7-8 



+_ — 

— ~+~ 



Considered also an aspect of coping by curriculum developers (See footnote, Table 1) 
Considered also an aspect of decision-making by curriculum developers. 
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c U rricui™ B implct°^ s £h ah^S? eS ^"V" J.r6gram., no enduring 

into jSiftfi' "'If'" "tUZS^FJV**^ 5 ^ * Ms ""art anVwai sustained 

the compares laf I J^bl^o "^V * feW V1 ' Sual illustrations of some of 
found Sn GrapSs | thrbufh 11 b?fftS!!?J!!? S 4" T ?-- 1e - ?' TheSe 11 ^^trations are 
indicated by P the X *ST symbols' xxf rP?pri ^fS- or / Xposure pa " erns are 
HLAY curriculum and tested Tu 5hrJ Li ^ t0 th0Se stud enis exposed to the . 
to those tilted til Itllt u t e years ( pure experimental"); 000 refers 

m?xed conditions XOX nTn^f* ^ " eve : ex P°" d ("pure controls"); and the 
onf of ?he other S years but S^'h th °V xp ° sed durin 9 the third year and 
occurred only in the you^gesS cohort Hs t ^ XXX conditl '°" 

requirements in the oldS ??JnS£ EfnS?' 5 " 6 ]' due T ^ v ariations in curriculum 
see what curr cul urn implct bccurrpd ?E-- S --- -- Iri Graph 6, for example, 
dimension, with tS too half of thp nlL? ** yeaPS Wlth aspect to the knowledge 
and the bottom half re? e ?r ng to the'oldlst [fv^? "Mrt H^est cohort < 4-5*6) 
HU ISSLSS-" 1 ?* a ^ Patt ? rn ^t'SnViSSJS L^-e 7 c1 } fro^° p a 3 e V d S ex° o u r re Ph t^ Ifi? • 

evln h jf 9 tS^ ma r mUm ^ y6ar three ' ?n ^ o ^ i 0 ' So) 

condi ion er6 S ^1^" Ideal "lat^rn^ ? 0 ' " 6 ^ P3C6 w1th that - the°ex erimental 
curriculum impact 2ch^ 

not statilti^liv ^nn I- J th u dlfferen£ es among the three conditions are 

in the decisiln P - m aking prorlJs "hows 5 dffflrH^K r «e°nsi5U alternatives 
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significance has been reached in the differences across exposure conditions 
but not because of increases in "the two experimental groups. Indeed, all t.. 
groups have actually experienced systematic declines across the three years; 
but '-'? n wltn the declines, the experimental groups stayed ahead of the cont 



three 



group in each year. This pattern suggests that while peer and other influences 
in the early tean years may be counteracting the impact of the HLAY curriculum, 
tnose influences do seem to be blunted somewhat by curriculum exposure. In other- 
words, while the youngsters during those troublesome years may.be inclined anyway 
to lose^some of the conforming and compliant tendencies they had as children, 
they will not lose as much if they are exposed to the HLAY program. 

*u This same general tendency to lose ground in the 6-7-3 cohort can be seen in 
the, bottom, half of Graph 9, which deals with the ability to generate alternatives 
in the decision-making process. While the pattern is somewhat erratic this time, 
especially in the second year, there is a definite drop in scores for the control 
group (000) from the rirst to the third year, while the drop is far less serious 
for the one experimental condition (XOX), and both experimental conditions in 
year three seem to be on the rise from year two. In thy top half of the same 
graph, for the 4-5-6 cohort, we see increases across the years in this skill for 
pom control and experimental groups, but the scores increase more for the experi- 
mental group. We think that these last two graphs (No. 8 and No7~9) point to the 
interesting observation that the HLAY curriculum can be expected to have its impact 
in somewhat different ways, depending upon the age groups involved, and upon the 
particular point in age at which youngsters are exposed to it. The so-called 
tempestuous teens," which start in the junior high years, can be expected to 
bring for nearly all youngsters a time of boundary-testing, some rebelliousness, 
and a general tendency, reinforced by the peer culture, to reject adult ways of 
thinking. In this period of life, perhaps the most that can be expected of any 
scnool-based prevention program is to slow down such tendencies, or to neutralize 
them somewhat. It is likely also that the earlier the prevention effort begins, 
tne greater will be its impact on the teen years. In that connection, one can 
see in these graphs that With very few exceptions the XOX exposure condition 
(exposed the first year but not the second) yields relatively stronger curriculum 
impact than the OXX condition (exposed for the first time only in the second year). 

In Graph 10 we have illustrated our findings about HLAY curriculum impact 
on attitudes for the 4-5-6 cohort, even though the evidence for impact here did 
not rate a plus on Table 3. In this graph, we are dealing only with the total 
score summing up curriculum impact across all the specific attitudinal areas — 
listed on Table 8. We have found repeatedly in this project that attitudes relating 
to alcohol use (like most other attitudes) are so hard to change, and/or that 
small changes are so hard to measure, that the total score is often the best 
indicator. The situation in Graph 10 is further complicated by the fact that 
our instruments used more elaborate attitude measures starting in grade 6 . so 
that the measurement basis in year three for the 4-5-6 cohort was different 
from that in years one and two. That may well be the reason, in fact, that the 
control/experimental difference for this cohort did not reach statistical 
significance in the third year, and thus did not rate a plus on Table 8, as it 
would have on the basis of the first two years. As Graph 10 shows, the gap 
between the experimental s and the controls widens from the first to the second 
project year. In the third year, the results still favor the experimental s , but 
not by as much with the more complex battery of attitude measures. 
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GRAPH 6 
CUMULATIVE PROJECT IMPACT 

• Grade Cohort 4-5-6: KNOWLEDGE 




CUMULATIVE PROJECT IMPACT 
B. Grade Cohort 6-7-8: KNOWLEDGE 



Range of 
Possible 
Scores 



id 

9 
8 
7 
6 
5 
4 
3 
2 
1 
0 

Treatment 
Pattern 
*N = 

Project Year 
School Grade 



x 
144 



0 

30 
1 
6 



o 

34 



8.58 




xb 
143 



ox 
30 
2 
7 



oo 
34 



XOX OXX 000 

144 30 34 
3 
8 
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GRAPH 7 




CUMULATION PROJECT IMPACT 
Grade Cohort 6-7-3: SELF-ESTEEM 



id 

9 
3 




Project Year 1 ( n.s.) 2 (n.s.) 

School Grade 6 7 



KITE: n.s. =» not statistically significant 



GRAPH 8 



CUMULATION PRQdECT IMPACT 
Grade Cohort 6-7-8: SELECTING RESPONSIBLE ALTERNATIVES 




NOTE: n.s. = not statistically significant 



GRAPH 9 
CUMULATIVE PROJECT IMPACT 
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A. Grade Cohort 4-5-6 
10 
9 



GENERATING ALTERNATIVES 



Po ss ib le 
Scores 



Treatment Pattern 
N = 

Project Year 
School Grade 




NOTE: h.s. = not statistically significant 



CUMULATIVE PROJECT IMPACT 
B. Grade Cohort 6-7-8: GENERATING ALTERNATIVES 



Range of 
Possible 
Scores 
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Treatment Pattern 

Project Year 
SefrooT Grade 
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GRAPH 10 
CUMULATIVE PROJECT IMPACT 



Grada Cohort 4-5-5: ATTITUDES (Total Score) 




School 

Grade 4 5 



NOTE: n.s. = not statistical 
significant 
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GRAPH 11 
CUMULATIVE PROJECT IMPACT 

Grade Cohort 6-7-8: PROBLEM DRINKING 



Possi bl @ 
Scores 



1.06 



Treatment Pattern 
N = 

Project Year 
School Grade: 



(no measure) 

1 
6 



xo 
140 



ox 
22 
2 
7 



oo 
31 



1-13 1.17 




XOX OXX 000 

138 28 29 
3 

8 



NOTE: In this graph, the smaller numbers indicate fewer 
related to alcohol use. ~ 
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. Finally, in Graph 11, we can see some limited evidence for HLAY curriculum 

impact on actual drinking behavior in the 6-7-8 cohort. Remember that this is 
the first or youngest cohort for which we have more than one year's measures for 
different aspects of drinking behavior (our test instruments introduced these 
starting only in grade 7, the grade or age at which significant drinking first 
starts for most youngsters). Note this time that the smaller numbe rs are the 
more favorable ones. That is, the lower decimal figures indicat e lower levels 
of problem drinking. Here again, in this graph, we can see the kind of "built 
in tendency to lose ground" with youngsters as they move into the junior high 
years. Problem drinking of various kinds is admitted by the youngsters in all 
Jnree of the curriculum exposure conditions, However, those in the XOX experi- 
mental group increase in their problem drinking levels much less than those in 
the totally unexposed control group (000). The other exposure pattern (OXX) 
doesn t make out so well. While still below the control group at the start it 
catches up to virtual parity ir the third year. This difference- between the two 
experimental conditions indicates again that early curriculum exposure, before 
the junior high years (XOX), produces a more favorable prognosis than later 
Th P °5"« (0X :K even if ex Posure takes place two years out of three in both cases. 
The differential between the early-exposed (XOX) and the never-exposed (000) in 
the two years shown in Graph 11 is quite substantial. 

A J 1 * in -! 11 ' the long-term impact of the HLAY upon cohorts of younasters 
KEh S hv SJf over a 3-year period is encouraging at several points, as indicated 

* y I II *u and by , the T^ustrative graphs that we have just reviewed. The 
oldest of the three cohorts analyzed here (6-7-8), which spanned the crucial 
years of the entry into adolescence, showed especially promising results in some 
of the affective and behavioral dimensions of program outcome. On the other hand, 
it must be admitted that evidence for curriculum impact was not in any sense 
systematic or across the board; it was the exception rather than the rule, when 
we consider all of the grade levels and all of the various dimensions of the 
curriculum. Also, the Ns for some of the control and mixed conditions in the 
older cohort are quite small, although that situation does not impair the 
credibility of our conclusions nearly as much with longitudinal as it would with 
cross-sectional analysis. Finally, our readers should understand clearly that 
wharwe have presented here regarding lono-term curriculum impact is not con- 
clusive, for much additional analysis remains to be done by means of truly 
longitudinal designs across the various pane. 3 of our data. Plans for these 
more complicated analyses, and the additional information we would expect them 
to yield, are addressed to some extent in the Scientific Appendix to this report. 

MAJOR FACTORS AFFECTING CURRICULUM IMPACT 

Determining whether or not a given curriculum or other prevention program will 
work is not really so simple a matter as comparing the test results of students 
who are or are not "exposed" to it. Many factors intervene between the initial 
design of a program and its "outcome," however that is measured. One important 
intervening variable is whether or not the program is itself soundly conc eived- 
Bo the elements it contains really have anything to do with the problem it is 
trying to ameliorate? If so, how much? Applied tc the HLAY program here under 
scrutiny, such questions would take these forms: Are there empirical grounds for ' 



. , . — _ ~ r _. ^ . a ,,, young 

people s lives, such as parent and peer influences, demographic traits, and so on? 
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a A second obvious question that has to be considered in assessinq how well 
traSlISi T£* S . is how well it was "delivered." In the prelS" St!S«rthat 
uSnt! 5 m*2£ nfTf - ° f qU 2 St10nS about the ^achers, who were the chief 

t i h , e | V ? : H °? were the teachers trained? How much good did 
to Jhe'r cfa^rooml5 em L H0 * ^ fitment to the HLAY program did they bring 
a?cordan C P with ^^^.""^^n^ous were they in implementing the program in 

? 5 " t tra l niT] V they had received? What difference, if any, did 
eva ul'ion 1" L e3Cher c^racteristics make for how well the students did on the 
Sfi^iii?S*Sl^k-"i 1 ?^5ts.?_ While we still have much analysis to do on all 
alona q at S? \ f ^chers, we do have some relevant information to pass 
UrSrtlh IhnnflS** ™- ] f l rst ' however, address the questions in the above 
ftSlf 2 2Sn^VS le ? anCe ° f the Van " ous dimensions of the HLAY curriculum 

culum l2eif ;«J3 4 SS a J-- ?ri pUt a ? other wa ^ how much "potential" the curri- 
culum itself contained for making any impact on drinking. 

The Potential for Curriculum Impact on Drinking Behavior 

^n^M! 1 ' 1 ! b f clear from our discussion above, and from Table 1, our data- 
collection instruments contained measures for a variety of factors influencina 

S"3iftV n - Mi V- M t0 the HLAY ^ciil™. Thlse othf? ffciSS ?SI aded 
sblill?l?na 1iSJHfS P ?Jj var i ab l es l^e age, sex, and race, as well as the major 
Jp n -ike parents, peers, and religion. All these other factors 
with a f lS;Sf on a child's life before, during, and after his/her experience 
with any school program. Therefore, in evaluating such a program, it is important 
to determine how much "room" there is for the program to have an nfluence! given 
ZL P JZ f.. the !° cial and demographic factors, already at work, which have 
rZtnT^l a L 1t 1 We [ e ^ mU ^ h of the influence that will determine how a young 
?hl 2i n a? Wlt f alcoho1 - One way of estimating the relative potential impact of 
fjj"^ curriculum, compared to all these other influ ences, is to throw them all 
thl H^nHonf 5 ^".. 6 ^ 3 ^ 0 "' wl "th various kinds of drinking behavior measures as 

n 11 ou tcome" variables. Table 9 shows the results for this method 
^ih^nTlc g fTu Q p °t e "ti?l for curriculum impact, based upon our cross-sectional 
aS* £E ^ ™! . sub f am Ple S> varied in size somewhat because of occasional missing 
data or non-response to specific measures.) 

nr 3 Jnntc e n?^ f • ^ K B l*-9-f§y De interpreted simply as the net proportions 
^amoilEts of the variation in the different measures of drinking behavio r that 
can.be accounted for by knowledge and information about alcohol .self-esteem 
decision-making skills, and attitudes about alcohol, after removing the effects 
ot social and demographic variables. The so-called "curriculum variables," whose 
tl?£t H a lt I if Li J5 is table, are the cognitive and affective traits of the 
j£S?Js that the HLAY program is aimed at influencing. Quite aside here fron, the 
2« H wnether or not that program really does influence those student traits, 

we are addressing the question Of how much change in the (aggregate) drinking 
behavior of students could the HLAY program make, even if its impact was at 
maximum levels. As the figures on this table indicate, this potential for 
curriculum impact ranges from a low of .04 (4%), for Grade 7 monthly quantity 
fnd 2252 H S? o f r drinfc ;" n 9, to a high of .26 (26%), for Grades 10-11 annual quantity 
and frequency Generally speaking, the figures are modest, indicating that when 
12*1* family peers, religion, age, sex, race, and many other things are 

taken into account, there is a limited amount of room left for any alcohol educa- 
tion or prevention program to have much effect. The fiqures are smallest for the 
lower and junior high grades, then they reach a peak, in general, in the central 
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TABLE 9 



INDEPENDENT CONTRIBUTION OF TOTAL SET OF CURRICULUM VARIABLES TO DRINKING 
BEHAVIOR OF STUDENTS NOT PREVIOUSLY TESTED OR EXPOSED TO CURRICULUM 





br 4 


GR 5 


GR & 


GR 7 


GR 8 


10-11 


Gr 1 


Range of N 1 s for thpqp Anaivcac 

• •^v. w« ii ^ i ui liicjc Mild 1 V5S5 
















Maximum 


685 


239 
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541 


118 


251 


Mi nimum 

' 1 1 1 1 1 1 1 1 Mill 
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480 
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Childhood Drinking Behavior 
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.10 
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.12 


.08 


.12 


.12 


rfuuiein unnKing 
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.14 


Current Drinking Situation 
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.12 
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Expectations about Drinking 
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Monthly Frequency 
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.09 


.24 


.17 


Yearly Frequency 








.12 


.09 


.21 
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Quantity 








.07 


.05 


.19 


.09 


Quantity-Frequency Monthly 
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.06 


.23 


.19 


Quantity-Frequency Yearly 
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Note: The data for this table come from the unexposed or "naive" students 

LTJ h 0 °In P ? Table 3 ' The severe attrition in Ns from 

!!! J \ °u - S table are accented for by the "list-wise deletion" 
procedure built into our regression program, which has the effect of 
eliminating^from the analysis any and all students who did not respond 
to an of the questionnaire items. The variation in Ns from maximum 
to minimum in the above analyses is due to still further attrition 
™„ an ? subj ?ct s failure to respond to all items constituting a 
m,n?? ^L Vanab I? ; consequently, the effective Ns are smaller for the 
multi-item variables above than for those measured by a single item. 
The R* values were obtained for the set of knowledge, self-esteem, 
decision-making, and attitude variables after partial! ing out all 
other variables (demographic arid non-curriculum) measured in this 
PrPJecti Grade 9 is omitted from the table because the Ns were rather 
15 \ JJ * & mi ^ analysis essentially requires that each student 
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ft ?h SXS ye r^ S °r d by a declihe ^ain during the senior year of 

5 e r fl 4t ^ el ? high Potential for curriculum impact on drinking 
f?SS2 fJ!. 9 ?^? 7? a " d 11 is somewhat ironic and frustrating in view of our 
til?!! 9 *! J!- : I""1 ? - } that f ctual curriculum impact on the cognitive and affective 
att tudL Im'SS Hi?*? t0 0CCur 1n the earlier grades (especially 

kill iri 2iS tSf h In exami " lng Table Si^iveri it is important to 

rnnS J„, !j t a V he ri . gu r es ar<e <l u i te parsimonious estimates of the 

room for potential. curriculum impact. They are the net residual figures gen- 

Hr?^?nnK^ 9r - SS1 ? n equations from which other plausible influences upon youthful 
o^e^ wnrnf a ^L r ( social a "d demographic)-hiV? already been eliminated. In 
after a variJS nf 6S ^ ates ° f what is left for the curriculum to impact, 
?Sto i?4.mi ^L 2 !^ .^H^nge^on the youngsters' lives have been taken 
even 10% ? la fl ™ 1 %S ,n ^ of ^™> though the figures do not often reach 
fl ii 10 . or mo r?)» they are by no means negligible and should be understood 

Buiff y supporting the theoretical basis on which the HLAY curriculum is 

Graphs can provide somewhat more visual illustrations of the relative 

potential we have been discussing. In Graph 12, we can see that the factors 
addressed by the HLAY curriculum (knowledge, self-esteem, decision-making, attitudes) 
could affect 7% of the problem-drinking in grade 7, out of the 35% that the total 
equation could account for. At grade 8, the corresponding figure is 5% out of 
16*| at grades 16-11 ,11% out of 35%; and at grade 12, 14% out Of 38%. In the 
last two graphs (13 and 14), quantity and frequency of drinking on a monthly and 
an annual basis, respectively, we see the same general pattern with respect to 
the relative potential of the curriculum-related influences. However, in these 
two graphs we can see a striking increase* after junior high school , in the amount 
of variation in drinking patterns that can be explained merely by the combined 
social, demographic, and curriculum-related faetors (about 50% or more). Also, 
the cognitive and affective factors addressed by the HhAY curriculum now account 
for much more of the variation, and therefore of the potential for program impact. 

Teachers and Curriculum Implementation 

Altogether, across the three project years, 430 different teachers were in- 
volved in providing our data, of whom about 3/4 represented "experimental" 
classrooms and the rest "control" classrooms. Our various evaluations of the 
teacher-training component of the HLAY program have indicated that in general 
the training has a definite impact of the desired kind in getting teachers ready 
to employ the HLAY materials and philosophy in their classrooms (see Rankin, et 
a]., 1378, and Tarnai, et al., 1981). At the same time, however, teacher attitudes 
toward various aspects of alcohol use did not yield to training as much as most 
other teacher traits did. Furthermore, the outcomes of teacher training were 
affected by certain pre-existing factors, such as the amount of classroom ex- 
perience a teacher had had and the level of enthusiasm which the teaeher brought 
t? the training workshops. We should not be surprised that training affects 
different teachers differently, or that teacher attitudes toward trainina and 
toward alcohol should account for many of the differences in training outcome. 
Nor, in view of all that, should we be surprised to find, as we did, that in 
their actual employment of the HLAY curriculum in the classroom, teachers had 
less overall impact on student attitudes than upon any other single dimension 
of the curriculum. Indeed, in the follow-up questionnaires that we administered, 
teachers tended to rate the attitude-oriented parts of the HLAY curriculum as 
among the least important parts. Social scientists know that attitudes are 
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GRAPH 12 



COMPARATIVE RELATIONSHIP OF CURRICULUM 
VARIABLES AND OTHER VARIABLES WITH PRINKING BEHAVIOR 

(Figures are Percents of Variance Explained in Drinking Behavior) 
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COMPARATIVE RELATIONSHIP OF CURRICULUM 
V ARIABLES AND OTHER VARIABLES WITH PRINKING BEHAVIOR 

(Figures are Percent! of Variance Explained in Drinking Behavior) 
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G.8APH 14 

COMPARAT IVE RSLATI ONSHIP OF CURRICULUM 
VARIABLES AND OTHER VARIABLES WITH DRINKING BEHAVIOR 

(Figures are Percents of Variance Explained in Drinking Behavior) 
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s?L%^£'nfl^W y l° r aRy s1 " 9nl T cant P eHod of time, and even more 
aSbi^afent 96 a 9* nts (teachers, in this case) are themselves 

, N 11 * aspect to the actual implementation of the HLAY curriculum in the 
classrooms, there were at least three fairly mechanical factors we attempted 
to measure that car, usefully be discussed here: {1) The number of classroom 

condurU' c ™ lu Vxposure (° ut ° ? the 15 called for) that teachers actually 
£ i?) The relative amounts of emphasis actually given to the various 

JS^MKhS /^ r r^ lum: knowled 9e. self-esteem, decision-making or coping, 
»ni» Jh S%; nd ^i.The amount_pf time that was permitted to elapse between 
S5 Si IhSTw! -^rriculura and the administration of the test instruments. Our 
tltfJn h th «e factors came from brief reports that teachers filled out and 

f?2E iS rSiS°Si^M f StUdent tests - We wil1 address each of these factors 
rirst in a brief descriptive way. 

UdbJCums ^•■° f -^ U - ri ^ u1um -. exposur ? : T - fiere enormous variation across the 
the ] 
peril 

fome uiqm at ucners: n we count at least 14 periods as "virtual 

ShIr" C |^ W ; h th , the ? tAY goa l' then we can say-that project-wide half of the 
teachers met that goal at grades 4, 7, and 9, while at grades 10 anTTl that 
goal was met by two- thirds of the teachers. At the other grades we studied, 
compliance with the goal of 15 (or 14) periods of curriculum exposure was reached 
by a third of the teachers or fewer; indeed, at the 12th grade level almost no 
teachers spent more than 10 periods. 




Bwereatia-I emphasis on various dimensions of th P curriculum: Each time the 
teachers taught with the HLAY curricu lum in their classrooms, we as ked that they 
Jnn i a | S ° n ^questionnaire how much emphasis they gave (respectively) to the 
Knowledge, self-esteem, decision-making, and attitudes dimensions of the curriculum 
me response categories for each dimension were "much," "some," "little," or "none " 
admittedly a very rough measurement system. As one might expect (and we expected),' 
the extremes ("much" or "none") were more readily intertable categories than 
, WS m?h 5 intermediate ones. However, we found that almost no teachers anywhere 
would admit to having given any of the curriculum dimensions no emphasis. In the 
case or the knowledge dimension, the most common response was^mueh" emphasis, 
reaching close to 100% of the teachers, indeed, except in the lower gracU (& and 5) 
mat is not surprising, since this aspect of the curriculum is not only the most 
conventional part of alcohol education programs, but is also the easiest to teach 
\-or tne other three dimensions, the modal response category was "some" emphasis, 
though for self-esteem there was an approximately even split in grades 4 and 7 
wnere about 50% of the teachers were found each in the "much" and the "some" ' 
5f t 2^2r i S- s °/ emphasis. This same kind of even split occurred in grades 10 and 
11 both for decision-making and for attitudes. Otherwise, almost all teachers in 
an grades and for all dimensions claimed at least "some" emphasis. 

... Interval between curriculum exposure and testing : In the expectation that 
the performance of the students on our measuring instruments (tests) might be 
affected by any appreciable delay in the administration of the tests, we tried to 
keep trac* of this factor as a variable to consider in our evaluation. The main 
empirical break" in our data oh this subject came after a delay of three day 
that is, the majority of our teachers got their tests administered within thri 



__JSJ 

three 



o 4v 



ERIC 



35. 



gays or fewer of the completion of the HLAY curriculum. At some grade levels, 
Z" e Z e ll t ? ere wer ^Preciable numbers of teachers (and thus of classrooms) for 
wmch the lapse was considerably longer than three days. The usual reasons for 
S? 9 fr. u?«t S were: (1) A vacation break of some kind intervened before testing; 
Uj the HtAY curriculum materials and exercises were integrated (and therefore 
scattered) across a more general course or unit (like health education), the overall 
testing for which was done at the end of the unit; (3) Logistical oroblems inter- 
vened such as failures of the clerical staffs and/or of the postal service to 

SMcultE? S3 f4 ^SJS 1 ^- 15 J he teachers on or other miscellaneous 

teachers failed tn SSJhJK" 6 *"? 5 howeve ^ half «r more of the "experimental" 
teacners failed to administer tests within three days in grades 4 5 and 12 

Otherwise the three day interval (or less) was the rule in about 80% of the' 
experimental classrooms at all the other grade levels. " 

Hhi1c S uf i.^\ 1 !?^:^7??r:?,. t ^ 1 ^ jrt „ teacher Performance fo r student perform^ : 

the hsS MilJi I to complete a thorough and comprehensive multivariate analysis of 

25.12 P la ^ ed .^"Cher-related variables in curriculum impact on students we do 
mil some grounds for suspecting that teachers can make a different. Jul? how 
?eacherra e t re a n n' a J a °l eV ?:v ^ 'Stances, " not yet clear? torty 

dU^ifblul iSIrSlS U ? fail6 5 'u- 0 ™ way 0r another t0 ^Plement the HLAY 
SnZl^ £ a I °I l n !u- t0 des 19 n » and this could be one important reason for the 
the tatist?cal ^J^^^^^f 1 ^ m0st aspects. On the other hand, 
d???eSie2 JS luJK&i.S we have carried out so far do not show much systematic 
airrerence in curriculum impact according to teacher performance. 

Some additional evidence bearing upon the relative importance of teacher effort 
and commitment in the implementation of the HLAY curriculum is to be found in the 

oSrino th^;-"^ 6 dl ' d 0n that matter duHn S the third year of our Infect. 
the first t^j years of the project, we had tried hard both to monitor 

•tStiliS of'thr^ni? 16 f 1 " Sr0 2 m and to enhance teacher compliance with the par- 
ticulars of the implementation design: In doina so, we had had to relv Drimarilv 

?or°^p°; g01n ? Cor T uniCatl '° nS wUh the teachers-by mail and by tele phone 'except 
^oillr^SLn^^r^ ^ h6y had u b6en 91 ' V6n at the outset and in the subsequent 
afiSlSi / IS °! tra ,l nin g at the start of each new school year. Our long 
?on^;^Jr m t !%? ,tes . w ?? re the HLAY program was being implemented made it 

I cal 'y and financially prohibitive for any members of our research staff to 
3^?v^ SerVatl0 ? al V1SltS t0 the P^Ject classrooms and monitor the process of 
Slii?S^ r ---3 1 !? n - $ J 95 - flr? ? hand - H °wever, as it became obvious that teacher 
compliance with the implementation design for the HLAY curriculum would inevitably 
be quite variable across the project, eh would seldom be complete, we began to 
consider whether any more could feasibly be done to gain greater teacher commitment 

the reou ifg'iX?!!? T curriculum could more of ter. le. implemented tfi 
the requisite number of classroom exposures, the requisite efflohasis on eaeh eurri- 
™l u ™ component, and the appropriate teaching methods employed, as called for in 
ft£eS£ rn^JSI; -i 6 ^estioned too how much difference such an enhancement of 
teacher commitment and compliance would make in curriculum impact upon the students. 

... A J t^ suggestion of our colleagues at the. Educational Service District No. 
121, who had developed the HLAY program and trained the teachers, we selected a 
few teachers that they recommended to us as more skilled and committed to the 
program than most of the teachers. Special contracts were concluded with these 

^achers, in which we agreed to pay them $100. each for the strictest 
possible compliance with the implementation design of the HLAY curriculum, and 
I e l a n 9 / s ! nS / t0 i Ub ?in to a number of monitoring visits Co their classrooms 
by professionals from the ESD staff. Altogether, three of these selected teachers 
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representing four different gra 2 levels, entered into these contracts and 
generated enough data for analysis. We analyzed the data from their classrooms 
separately and made comparisons of their data with project-wide norms for 
evidence of differential curriculum impact on students in the immediate or 
short- term sense. 

To speak in generalities, we would have to say that the results of this 
compan son were mixed, as we have usually found the evidence on teacher impact 
to be. At the 4th grade level, there were no significant differences for students 
or the selected teaehers, either on a pre- test/post-test basis , or as compared 
with project norms for that grade, with the exception of some modest improvements 
over project norms in the area of decision-making, skills. In grade 5, the picture 
was but little better: test results showed significant gains by the students of 
the selected teacher on a pre- test/post- test basis in a couple of the attitu- 

dinal areas and in one of the decision-making skills; but even the post-test 

results did not compare favorably with project-wide norms for fifth-graders. In 
grade 6, the students of the selected teacher in general showed no improvement, 
either on a pre- test/ post- test basis, or by comparison with project-wide norms for 
tnat grade, with the exception of one of the decision-making skills, where the post- 
?n S ^f C ?n^ wer 5 cl " rly superior both to pre-test and to project norms. Finally, 
in tne 10th grade* the only secondary level at which we had any special or selected 
teachers, the students showed no evidence of having benefitted by any special 
teacher effort, either on a pre-test/post-test basis, or by comparison with project 
norms for that grade. Such evidence as there was, then, for the importance of 
strict teacher compliance with the main requisites of curriculum implementation, 
was more noteworthy in the elementary grades than at the secondary level. 



CONCLUSION 



ERIC 



This report has covered the most general findings of our evaluation of the 
Here s Looking at You" Alcohol Education Program during the period 1978 - 1981 
An earlier report, dated December, 1973, covered our evaluation of a field- testing 
phase, during which various formats of teacher training were assessed, measures 
were developed for evaluating program impact on students, and limited evaluations 
were actually earned out of student impact, on an immediate or one-time basis, in 
various grade levels and locations. This present report goes beyond the earlier 
one by covering evaluation research that was based on (a) an elaborate quasi - 
experimental design; (b) large quantities of student data; (c) a longitudinal 
component, in which some of the students were followed individually and measured 
repeatedly over a 3-year period; (d) consideration of important factors and in- 
fluences from outside the HLAY program itself, such as parents, peers and demo- 
graphic factors; and (e) some assessment of the HLAY program impact on actual student 
drinking behavior, not heretofore possible without the longitudinal data. 

We assessed the impact of the HLAY alcohol education curriculum on students, 
both on an immediate or short-run basis and over a 3-year period. In an immediate 
sense, after exposure to the HLAY unit only one time, curriculum impact on 
students was definitely found for all students in the area of knowledge or infor- 
mation about alcohol and alcoholism. In addition, We found that self-esteem was 
favorably affected by short-term curriculum exposure in grades 5, 6, and 7, and that 
decision-making skills of various kinds were improved for youngsters, starting in 
grade 6 and on Into junior high and high school . As we had always found in our 
preliminary studies, attitudes relating to alcohol use were least affected by 
exposure to the HLAY unit, but there was evidence in our research that students 
from grade 3 on down were changed somewhat toward ah attitudinal stance favoring 

« i 2 



37. 



ERIC 



moderate drinking (as opposed to abusive or excessive drihlnnaV 
curriculum. impact on actual drinking behavior could no t og ca i y be eva'fu^d 
HLAY^nit* °L thB behavi ° r /eported would have occurred pHor ^exposure tolhe 
ariLZl k h However ' we dld ^'nd reason to believe that high levels of pre-existina 

^^^^^^^m?^^^ the e HLAY° f cu P rri";um in9 
promisinq than anv ^ hfn 9 f on short-tenn curriculum imoact were more 

upon the quantity or freauencv of ,fr^n? ^ PSS^JrMjM. Dut not otherwise 

schooled pKvenliln nrSrfm, *]" ^ues.for srl*. intervention with 

»85? S? fences- EJ^i 9 in^ f5° 01 • 1nf iUenc " actuall y lost ground compared to 

£? f ^V" K5Sy"SiM Xpecfa^rrire.^r fitSl* 
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the dpJp^nlnt J5 e ^! n .1u eSt . an , d s "? tained eff ort and expense that went into 

Wi mm SeT? ask whv ^^w^- k I^J t ^^P^^ i and ^to its evaluation, 
we_mignt wen ask why its impact was not more dramatic. Why didn't it "work" 

mi£ e d " na t could have been done (or could be done in future adopt onsHo 
already seen £?tL J, n J able 9 « and in the graphs which followed it* We have 
already seen that the HLAY program was well conceived, in the sense that it* 

are^af leas °7n ^^^^^ *W^WI**^™h*H*j 
lf>l'«lL ff™ E tne aggregate, clearly related to alcohol use and abuse. At 

behavior £S^2V^iM U ? ? 35 clear from those same graphs that drinking 
IS^ • 36 - r abuse) is not nearly so dependent upon these curriculum- 
nJr? n >,n^v n ^°- S as U is u P° n the other influences in the lives of students 
SSrS ^k ld"^??^!^ P S ri FiT ^ i0US influences. Even if the HLAY 
SIS ^? worked perfectly, there are stringent limits to the impact it could 

no°t n nLH q 9 |ble rS, a nd n f other influences. Still, that potential impact 

»f IK negligible, and it tends to increase, even relative to other influences 

if the cL^ n ?|LTan 9 d r0 affI^-- I^f ™£* S66m t0 be an ^cation here ?haT 
irthlri^d ISJfi VS h!2» affective traits addressed by the HLAY curriculum could be 

d 3 aI ' b * for t Junior high school, they might loom larger and more im- 
portant as determinants of drinking behavior later on. 

iu As l de u rom the potential inherent in the HLAY curriculum, the performance 
that h ^S5S!!i S in the classrooms is likely the most crucial factor % how fSlly 

nrnlarf h h can be rea 1 i zed . We were satisfied that the teachers in our 

according to thp ade 9 ua tely trained to implement the HLAY in their classrooms 
anH fniin ? design of the developers at ESD-121. Our procedures for monitoring 
and follow-up of teacher performance were as elaborate and effective a s We could 
make them, given the inherent practical constraints, budgetary? logistical 2nd 

£ P IS*" 51 * ? an r ° f u° Ur pr0dect teachers nevertheless proved unwili no or 'unable 
Ssi m ^f ent t ul ] y the program in their classrooms in accordance w the train ng 
23i S r || e l ved and tne commitments they had made. As frustrating a^ this 
are likliv S'S 5 y be from . an evaluator's standpoint, utricular, developers 
ffifli SSfJSLi s • m ° re f r us | ra ted, as they see the program they have so care- 
Si iL ?« ?S 6d b61ng compromised and attenuated by incomplete or slipshod implemen- 

maJ be MirfE'S*? " f9Ce 0f this Oration, curriculum designers 

may be inclined to "disown" their own programs, or at least the evaluations of 

f?rst fnSnV^ U~* eny thS val J dny ° f any evaluation that Is done without 

n 1 the program is being carried out meticulously according to 
™ JS\ • general, this is a sound position to take: there would seem to be 
n^ P 21- 1n evalua ting the outcome of a curriculum or any other program that is 
not being implemented properly. a 5 

nna Jn this particular instance, however, we have assumed a different position 
??^^--- ? i def S nSib 3 e at 1 ? ast under the circumstances of our particular evil ua- 
SiTiiE* 3 **' We u " der ? t00d charge from the funding agency as one of 
evaluating a comprehensive school-based alcohol education program under the 
laborato^v 2XBIK ° hta ^ ?0 P«&1ic school system, not under controlled 
ditinn* ILl « -fl 011 ^ -?o_be sure,.we wanted to do our evaluation under con- 
?32? f.SJi- J H m every classroom, approach as closely as possible the 
ISIJ I K and teacher behavior envisioned by the curriculum designers. How- 
departuS rSL f iS "i!!i 1n9 .4 PhaS ! ° f 0Ur WOrk ' we k new that there would be many 
?22J£^ n the u ldea J and much ^evenness across the project in teacher per- 
formance. Once we had chosen our schools and classrooms, we were "stuck with" 
whatever teachers went with them. After a period of try nq various wavs to get 
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« much teacher compliance as possible with the desiqn for curriculum implementation 
SarhI™S?- d With ° Ur plan for outcome evaluation, knowinq fufl wel ?hat ' 
l?h?5i r ?° m P lia "« was going to be problematic throuahout the project, or even 
We wou Id taS^fS* a I. C6 - ; -^-^ ^ fth.the following understandings? (1) 

S€ 5 fill ^MF^F M "^^^^ 

StaS *2X l5 - a ^ ^luate curriculum outcome under "normal condit Ons" ih a 

g S -^1 M'- W new school program hav? to Wtes ;ted, 

program design? differential teacher commitment and compliance to 

this Sr§2ifS"ir2ffir'"^ ^..feel^would have made little sense in the case of 
SJiiiSirS -IW r °J ect - Qn e opt^n might have been for us to oostpone outcome 
ill 1?| 8 J2* IL2LSE; 2! 2^ ed f ™ °«T Process evaluation jhafwe hafat 



the teacherT in «V 61 t classrooms, or in what we were doing with 
IkIA * rs, or in some other aspect of the implementation process durina the 
*£lS l!t rS ° f Research, ^suming we would have had the power to do so This 

Xavlmadrird a ?^i C h 2rrtn' h r aluati - n9 . a r nsuntry cm>w w&'jfti* 

k L t ■ dlff l cu 't to be sure just what we Were evaluating— version a version 

evaluate °and ^^§1^1 Had th ^ yearS of fund ^ * outcom ^ 
evaluation, and the time had come to assess program impact, in the aeareWt* — a* 

typical staff and classroom conditions. 

nr rn^!^ 0 r.. eViae ^ e ,. S ° far does not snow that variations in teacher performance 
?n SS,—?"- nonnally occurring in school settings make much of a difference 

SudeTte lf^\Z^^ r i?^W- m ^ my yet determine thaTsome 
+« Shi ui S5 sco r es . are affected by the sheer number of classroom periods devoted 
to the HLAY curriculum, or by the length of the delay before testing or oerhaos 
Ur ffifi n' P6CtS °l t ? acher performance or background. S?ven the limiLd potential 
behavior ttSJ&E? 0 f^E"** C ° mpete W1 ' th other influences in sSapfng^tuden? 1 
the HLAY'ifSSsf Sisfl?"- - - n any C3Se behoove an * school system which adopts 
thorough Jh--— ? =?- P " pr ° 9 r am t0 9 ive as much attention as possible to 
thorough and conscientious teacher implementation of the program in al iS a.nprtc 
if students and parents are to get at least the program 's^blst shSt " P * 
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